Data storage systems are arrangements of hardware and software that typically include multiple storage processors coupled to arrays of non-volatile storage devices, such as magnetic disk drives, electronic flash drives, and/or optical drives, for example. The storage processors service storage requests that arrive from host computers and that specify files or other data elements to be written, read, created, or deleted, for example. The storage processors run software that manages incoming storage requests and performs various data processing tasks to organize and secure data stored on the non-volatile storage devices.
Previous data storage systems have used background processing to perform specific data services. For example, some previous data storage systems have opportunistically performed certain data services “in-line”, i.e. on data specified by I/O write requests received by the data storage system, prior to sending the data to an internal file system within the data storage system (referred to herein as a “lower-deck” file system), from which the data is passed to the non-volatile storage devices. Data services that previous systems have attempted to perform “in-line” have included data deduplication, compression, and/or encryption. Previous systems have performed these services “in-line” opportunistically, in consideration of the current load experienced by the data storage system, such that the data services are performed only when sufficient operational resources of the data storage system (e.g. storage processor CPU, storage processor I/O resources, etc.) are available for the data services to be performed without adversely impacting the overall performance of the data storage system. At other times, when the current load results in insufficient resources being available for these data services to be performed in-line, some data specified by I/O write requests has been sent to the lower-deck file system without data deduplication, compression and/or encryption. Background processing may then be used to perform data deduplication, compression and/or encryption on data after the data was sent to the lower-deck file system.